Quantifying the separability of data classes in neural networks

We introduce the Generalized Discrimination Value (GDV) that measures, in a non-invasive manner, how well different data classes separate each given layer of an artificial neural network. It turns out that, at end training period, GDV L attains highly reproducible value, irrespective initialization network’s connection weights. In case multi-layer perceptrons trained with error backpropagation, we find classification complex sets requires temporal reduction class separability, marked by characteristic ‘energy barrier’ initial part GDV(L) curve. Even more surprisingly, for set, is running through fixed ‘master curve’, independently from total number network layers. Finally, due to its invariance respect dimensionality, may serve as useful tool compare internal representational dynamics networks architectures architecture search or compression; even brain activity order decide between candidate models function.